Building a model-based reasoning framework for early analysis and prediction or responsiveness

A key quality concern in building mission-critical systems is responsiveness; Responsiveness comprises the total end-to-end latency of multiple functional flows that can occur within a system. Thales Netherlands B.V. designs and develops many mission-critical systems, with the “Combat Management System”, that provides “command and control” capabilities to naval vessels, as a prime example. This system is being developed at the “System Definition” business unit, that is responsible for defining large, responsive and complex combat software. It is important that system engineers from this business unit can verify the expected performance behavior “on-the-fly” i.e. before the system is actually build. However, within the current system engineering process, performance assessment is performed too late e.g. only after the integration phase. This induces a large financial risk on the developed system, because not meeting performance requirements lead to cost overruns that can result in project failures. Therefore, performance assessment should be considered early during the overall process. This research thesis presents an integrated performance engineering method that can assess and evaluate responsiveness early; It uses responsiveness-related information that is currently scattered among Thales Netherlands B.V. and their subcontractors. Integration is achieved by defining a model-based process that makes use of current development artefacts (models) and by providing automation to system engineers in the form of modern tools. These tools allow the system engineer to compose systems and to predict the expected performance behavior at the same time. Structured performance engineering methods requires the existence of metamodels; metamodels describe the structure of systems and can provide a domain-level overview of all important responsiveness concepts that influences the performance behavior of the system. Therefore, this thesis developed two UML metamodels: one for describing the exact structure of systems and one for describing important responsivenessrelated quality attributes. By combining both metamodels, a (simple) analytical model can be obtained that can closely estimate the expected responsiveness behavior in practice. However, analytical models are closed-form solutions: the calculated results represent averages in the form of constants. It is also valuable to verify the certainty of the outcome. Therefore, a simulation-based approach is incorporated within this thesis, as simulations can predict system behavior by using distributions instead of constant values. As a result, the total end-to-end latency of a functional flow cannot be described only by a constant value (e.g. 500ms), but also with a degree of certainty (e.g. a peak latency of 500ms will occur in less than 5% of all cases). This is very important for requirement verification. Model-driven engineering techniques are used in order to “close the gap” between system models and (formal) analysis models with the goal of generating an executable simulation model out of UML system analysis models. The “proof of concept” tool developed during this thesis uses the IBM Rational Software R © product family; It is based on the Eclipse R © framework and provides good UML modeling capabilities, an integrated development environment and support for adding and extending existing UML functionalities by means of stereotypes and tagged values. Model-based approaches are supported by the provided JAVA transformation engine. Thales Netherlands B.V. (will) use the UML and the IBM Rational Software R © product family for all kinds of system engineering activities. The tool is available as two IBM Rational Software R © plugins: one that provides modeling support to system engineers and one that provides transformations that can calculate and generate a simulation model out of a composed UML system analysis model. This simulation model can be fed into a discrete-event JAVA simulator, that generates histograms with end-to-end latencies that can occur within the composed system. System engineers use this feedback for composing an optimal, responsive system configuration.